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MEASUREMENT OF THE ERRORS OF SERVICE ALTIMETER 

INSTALLATIONS DURING LANDING- APPROACH 

AND TAKE-OFF OPERATIONS 

By William Gracey, Joseph W. Jewel, Jr., 
and Gene T. Carpenter 

SUMMARY 


The overall errors of the service altimeter installations of a 
variety of civil transport, military, and general-aviation airplanes 
have been experimentally determined during normal landing-approach and 
take-off operations. The average height above the runway at which the 
data were obtained was about 280 feet for the landings and about 
440 feet for the take-offs. 

An analysis of the data obtained from 196 airplanes during 415 
landing approaches and from 70 airplanes during 152 take-offs showed 
that: 


1. The overall error of the altimeter installations in the landing- 
approach condition had a probable value (50 percent probability) of 

±56 feet and a maximum probable value (99*7 percent probability) of 
±159 feet with a bias of -t-10 feet. 

2. The overall error in the take-off condition had a probable value 
of ±47 feet and a maximum probable value of ±207 feet with a bias of 
-55 feet. 

5. The overall errors of the military airplanes were generally 
larger than those of the civil transports in both the landing-approach 
and take-off conditions. In the landing-approach condition the probable 
error and the maximum probable error of the military airplanes were ±45 
and ±189 feet, respectively, with a bias of +15 feet, whereas those for 
the civil transports were ±22 and ±96 feet, respectively, with a bias 
of +1 foot. 

4. The bias values of the error distributions (+10 feet for the 
landings and -55 feet for the take-offs) appear to represent a measure 
of the hysteresis characteristics (aftereffect and recovery) and friction 
of the instrument and the pressure lag of the tubing-instrument system. 
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INTRODUCTION 


In reference 1 the overall errors of airplane altimeter installa- 
tions in the landing-approach condition were estimated from an assess- 
ment of the various individual errors (instrument, pressure, and opera- 
tional) which contribute to the complete system error. In that analysis 
the allowable tolerances for each of the individual errors (taken for the 
most part from ref. 2) were assumed to represent maximum probable values; 
that is, values having a probability of 99-7 percent. The maximum 
probable values of the individual errors were then combined to produce L 

an overall error having a probability of 99*7 percent; that is, an error 1 

that would not be exceeded in 997 cases out of 1,000. 

The two individual errors having the largest values were the altim- 
eter scale error and the static-pressure error. For the analysis in 
reference 1 the values of these two errors were assumed to equal the 
tolerances specified for altimeter systems in military airplanes prior 
to about 1954; that is, an altimeter scale error of ±50 feet and a 
static-pressure error (based on an assumed approach speed of 150 knots) 
of ±58 feet. The combination of these values wiuh the values of the 
other instrument and operational errors given in reference 2 produced an 
overall error of ±170 feet. 

In view of the magnitude of this error and :.ts serious implication 
for the safety of landing operations under low -visibility weather condi- 
tions, a test program has been conducted to detemine experimentally 
the overall errors of the service altimeter systems of a variety of 
aircraft during routine landing operations. As an extension to this 
program, tests were also conducted to determine the altimetry errors 
during take-off operations. This report presents the altimetry errors 
as determined from 196 airplanes during 415 landing approaches and from 
70 airplanes during 152 take-off operations. 


SYMBOLS 


hg geometric height of wheels of airplane above runway, ft 

hp pressure altitude at height h g , ft 

hj_ pressure altitude indicated by altimeter with barometric dial 

set to current altimeter setting, ft 

Ah overall error of altimeter installation, h^ - hp, ft 
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T s standard atmospheric temperature at elevation of runway, 

°F abs 

T a actual atmospheric temperature at elevation of runway, 

°F abs 


APPARATUS AND TEST METHOD 


The overall errors of the altimeter installations during landing- 
approach and take-off operations were determined from a comparison of 
(l) the altimeter indication as read by the pilot when the airplane was 
directly over a ground station and ( 2 ) the correct pressure altitude at 
the elevation of the airplane as determined from the geometric height 
of the aircraft above the ground station. Prior to each test run the 
pilot was asked to set the barometric adjustment on the altimeter to the 
current altimeter setting as reported by the airport control-tower 
operator . 

The ground station was located near the middle marker of instrument 
landing system installations at two commercial airports serving civil 
transport, military, and general -aviation airplanes. For this investi- 
gation the average height of the aircraft over the runway plane was about 
280 feet in the landing-approach and kkO feet in the take-off operations. 
These heights are noted to indicate that the airplanes were a sufficient 
height above the ground for the static-pressure systems to be unaffected 
by ground effect. 

The geometric height of the airplane above the ground station was 
determined by photographing the airplane with a camera located at the 
station. The camera used for these measurements was a single -exposure, 

5 - by 5 -inch aerial camera mounted on a tripod with its optic axis 
vertical. The camera was equipped with a simple sighting device by 
means of which the camera operator could determine when the airplane 
was alined with the optic axis. (See fig. 1.) In most of the test runs 
the speed of the airplane was sufficiently low for the operator to 
"center" the airplane in the film frame. The camera record and the 
pilot* s reading of the altimeter were synchronized by means of a radio 
signal which was transmitted by the camera operator at the instant he 
photographed the airplane . 

The geometric height was computed from the length of the wing span 
on the photographic film, the actual wing span, and the focal length of 
the lens. The focal length of the lens was calibrated prior to the test 
program by photographing vertical ground markers a known distance apart 
and a known distance from the camera. This calibration also provided 
the corrections to be applied to the film measurements for those cases in 
which the airplane image was offset from the center of the film frame. 
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The geometric height computed on the basis of the wing span repre- 
sented the height of the wing-tip plane above the camera. The quantity 
needed for comparison with the altimeter readings, however, is the 
pressure altitude at the height of the wheels of the airplane above the 
runway. The basis of this statement is the facu that, when the baro- 
metric dial of an altimeter is set to the current altimeter setting, 
the altimeter should indicate the elevation of -:he runway when the air- 
plane is at rest on the runway . For this reason, the geometric height 
of each airplane above the camera was corrected to the height of the 
wheels of the aircraft above the runway. (Actually, the altimeter 
setting is computed for an altimeter at a height of 10 feet above the 
runway surface $ thus, an additional error will be present in the altim- 
eter readings of those airplanes for which the height of the altimeter 
above the runway is different from the 10-foot standard.) 

In the computation of the geometric height it was necessary to 
assume that the wings of the airplane were laterally level. From ground 
observations of the airplane over the camera station it appeared that in 
no case were the airplanes banked more than For an airplane with a 

wing span of 100 feet at a height of 300 feet above the camera, the 
error in geometric height introduced by a bank angle of 5° would be 
about 1 foot. 

The geometric height of the wheels of the airplane above the runway 
was converted to the corresponding pressure altitude above the runway by 
means of the equation 



where h p is the pressure altitude above the runway, h g is the 
geometric height of the wheels above the runway, T s is the stan dar d 
absolute temperature for the elevation of the airport, and T a is the 

actual absolute temperature at the runway. The elevation of the runway 
was then added to the pressure altitude h p and this sum was used as 
the basis of comparison of the altimeter indications over the camera 
station. Strictly speaking, the temperatures gi/en in equation (l) 
should be stated in terms of the mean temperatures (actual and standard) 
of the air column between the ground and the airplane. However, for 
the relatively low airplane heights (280 to 440 feet) and the small 
ground- temperature deviations from standard (-27° to +130 f), of the 
present tests, the error introduced by this approximation is negligible. 
Errors in height due to errors in the measurement: of ground temperature 
were also small. For example, for an error of 1 ° F, the error in the 
computation of a height of 300 feet would be less than 1 foot. 
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The airplanes for which altimetry errors were determined in the 
landing program included 7^ civil transport airplanes (operated by three 
scheduled airlines), 91 military airplanes (operated by various air force, 
army, coast guard, and navy squadrons), and 31 general-aviation airplanes 
(most of which were of the private-owner, as compared with the executive- 
transport, type). These airplanes represented 8 civil transport types, 

13 military types, and 18 general-aviation types. The airplanes varied 
in size from small private-owner types to medium-size civil and military 
transports . 


ALTIMETRY ERRORS IN LANDING -APPROACH AND TAKE-OFF OPERATIONS 


For an airplane in the landing-approach or take-off condition, the 
difference between the altitude indicated by the altimeter and the true 
elevation of the airplane is determined by the error in the pressure 
measured by the static-pressure source, the mechanical errors of the 
altimeter, the errors due to a tubing lag or to leaks in the static- 
pressure system, errors in the measurement and reporting of the altimeter 
setting, errors in setting the barometric pressure scale and in reading 
the altimeter, and variations in pressure altitude due to deviations of 
the atmospheric temperature from standard. In this report the overall 
error Ah of an altimeter installation is defined as comprising all the 
above errors except that due to variations in atmospheric temperature. 

The static-pressure error on a particular airplane should, for a 
given speed, pressure altitude, gross weight, and aircraft configuration, 
have a fixed value. However, since any or all of these quantities may 
be different during successive take-off or landing-approach operations, 
the static-pressure error may be expected to vary to some extent during 
repeated runs at the same elevation. These variations in static-pressure 
error, however, should not exceed the allowable tolerance specified for 
civil transport aircraft (-50 to 20 feet for the older aircraft, ref. 3, 
and ±30 feet per 100 knots at sea level for the more recent types, 
ref. 4) and for military aircraft (±25 feet per 100 knots at sea level, 
ref. 5)* It may be noted that the static-pressure systems of private- 
owner airplanes in the general aircraft category are not required to 
meet any specified tolerance. 

The mechanical errors of the altimeter include the scale error, 
which should be relatively Invariant, and a number of smaller errors 
(temperature, friction, acceleration, and aftereffect and recovery - 
i.e., hysteresis and drift at the completion of a pressure cycle) which 
will generally vary in an indeterminate manner. (For the relatively 
constant rates of change of altitude of most landing-approach and take- 
off operations, however, the error due to friction would be expected to 
be always in a direction that would cause the indicated altitude to lag 
behind the sensed pressure altitude.) The scale-error tolerances at 
sea level for altimeters having the so-called n sensitive” mechanism is 
±20 feet for the older civil transport airplanes (ref. 6) and ±50 feet 
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for the older military airplanes (ref. 7)* The tolerance for altimeters 
having the newer "precision" mechanism and which are being specified for 
the newer aircraft is ±20 feet for civil airplanes (ref. 8) and ±50 feet 
for military airplanes (ref. 9). Again, it may be noted that there are 
no regulations governing the error tolerances of the altimeters installed 
in private-owner types of airplanes. From a knowledge of the type and 
age of the airplanes tested, it is believed tnat most of the altimeters 
from which measurements were obtained were of the "sensitive" type. 

Errors due to the time lag in the transmission of the source pres- 
sure to the altimeter depend on (l) the pressure and viscosity of the 
air, (2) the rate of pressure change at the pressure source, (5) the 
flow resistance characteristics of the tubing (which depend principally 
on the length and bore of the tubing), and (^) the volume of the instru- 
ments at the end of the static-pressure tubing. For the relatively low 
rates of descent of the normal landing approach, the design of most 
pressure systems is such that the lag error will be small. For example, 
for a system of two altimeters, two airspeed indicators, and two rate -of - 
climb indicators connected to a 50-foot length of 5/16- inch-bore tubing 
and a rate of change of altitude of 600 feet per minute, the lag error 
at sea level would correspond to an altitude error of about 5 feet. 

Errors due to leaks in the pressure system depend on the size and 
location of the leak in the system and on the pressure differential 
across the leak. Although both civil and military agencies prescribe 
maximum permissible leak rates for altimeter systems, these tolerances 
may not truly reflect the leak rates of service installations. In the 
test program reported in reference 10, for example, serious leaks were 
found in about 20 percent of over 100 service airplanes on which leak 
tests were performed. The effect of a leak can be most serious in pres- 
surized aircraft if the leak occurs within the pressurized area. Although 
a large number of the civil transport and military airplanes for which 
data were obtained in this investigation were- known to be pressurized, 
the effect of this factor on the measured errors was impossible to deter- 
mine because of a lack of knowledge of the leak condition of the systems 
and of the degree of pressurization employed during the landing approach. 

Errors in the measurement and reporting of the altimeter setting 
by the tower were determined from measurements of the barometric pressure 
at the camera stations at the two airports . The instrument used for the 
measurement of barometric pressure was a barograph having an accuracy of 
about +0.01 inch mercury. For each day of operation, sample measurements 
were selected from the barograph record for time periods when the pressure 
was unchanging. These pressures were converted to altimeter settings, 
which were then compared with the current reported setting. A comparison 
of 65 measurements showed the reported settings to have a standard devia- 
tion of 0.013 inch mercury (about 12 feet in altitude). 

Errors due to the pilot's missetting of the barometric scale or 
misreading of the altimeter in the present tests are, of course, impos- 
sible to evaluate. However, an indication of the extent to which the 
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pilots attempted to provide accurate information may be gathered from 
the fact that in most cases the altimeter readings were reported to the 
nearest 10 feet and in many cases to the nearest 5 feet. 

It is true that, if there were a time lag in the pilots' reading of 
the altimeters after receiving the signal from the camera operator, the 
error distribution would be biased. For a rate of altitude change of 
600 feet per minute and a lag of 1 second in reading, the error due to 
lag in reading would amount to an altitude error of 10 feet. If errors 
due to this source did occur , the overall errors measured in the present 
tests would be biased in a negative direction for the landing-approach 
condition and in a positive direction for the take-off condition. 

With the barometric dial set to the current altimeter setting, the 
deviation of the airplane from the true elevation due to a variation 
in the atmospheric temperature from standard is a function of the amount 
by which the actual temperature differs from standard and the height of 
the airplane above the runway. For a temperature deviation of 40° F 
from standard, the pressure altitude will differ from the actual height 
above the runway by about 10 percent of that height. Since, in the 
present tests, the height of the airplane above the runway was converted 
to the pressure altitude corresponding to the existing temperature, the 
overall errors as determined in this investigation will become more 
positive when the air temperature is below standard and more negative 
when the temperature is above standard. For a geometric height of 
200 feet, for example, the effect of variations in atmospheric tempera- 
ture (assuming no other errors) would cause the altimeter to indicate 
220 feet at a temperature of 20° F and 180 feet at a temperature of 
100° F . 


RESULTS AMD DISCUSSION 


The range of the overall errors measured on each type of airplane 
in the landing-approach condition is tabulated in table I. Since the 
altimeter systems of the airplanes in the civil transport, military, 
and general-aviation categories are controlled by different specifica- 
tions for the static-pressure and instrument errors, the data in table I 
have also been grouped according to aircraft category and to the operating 
agencies within each category. A large range of error for a given type 
of aircraft can result from differences in the instrument errors and 
from differences in static-pressure errors due to variations in the 
landing speed and aircraft configuration. Errors in excess of the appli- 
cable tolerances for a given aircraft category may reflect the degree to 
which the altimeter systems are maintained by the particular operating 
agency . 
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The overall errors of the altimeter installations of the various 
individual airplanes in the landing-approach condition are presented in 
figures 2 to 4 . In these figures the errors have been grouped in 
20-foot increments and plotted in terms of the number of errors in each 
class interval. 

Figure 2 shows the distribution of 444 errors of 196 civil transport, 
military, and general-aviation airplanes as determined during 415 landing 
approaches. (The number of errors is greater than the number of landings 
because In a number of cases readings were reported for both the pilot* s 
and copilot* s altimeters.) The average value of these errors is +10 feet 
and the standard deviation is 53 feet. The probable error (50 percent 
probability) is, therefore, ±56 feet (O.675 ulmes the standard deviation) 
and the maximum probable error (99-7 percent probability) is ±159 feet 
(5 times the standard deviation) with both errors having a bias of 
+10 feet. 

In figures 3 and 4 the distributions of the errors for the civil 
transport and military airplanes are shown separately. (The number of 
test points obtained from general-aviation airplanes was too limited to 
form a valid statistical sample.) For the l6l measurements obtained 
from 74 civil transports in 154 landing approaches the probable error 
is ±22 feet and the maximum probable error is ±96 feet. with a bias of 
+1 foot. For the 248 measurements of 91 military aircraft in 
226 approaches the probable error is ±43 feet and the maximum probable 
error is ±189 feet with a bias of +15 feet. The larger error of the 
military airplanes appears consistent, at least in part, in view of the 
larger scale-error tolerance (±50 feet) for military airplanes as com- 
pared with that (±20 feet) for civil airplanes. It may also be noted 
that, as discussed previously, the military tolerances for the altimeter 
scale error and the static-pressure error were used in the estimation of 
the ±170-foot maximum probable error reported, in reference lj this esti- 
mated figure compares favorably with the ±l89-foot value determined in 
the present tests for military airplanes. 

The overall errors determined during take-off operations are pre- 
sented in figure 5 - This figure shows the distribution of 176 errors of 
70 civil transport, military, and general-aviation airplanes as measured 
during 152 take-off operations. The probable error for this case is 
±47 feet and the maximum probable error is ±207 feet with a -33-foot 
bias. The number of measurements obtained fer each of the civil trans- 
port, military, and general -aviation categories was insufficient to per- 
mit a determination of the standard deviatior for the separate aircraft 
groups. However, it may be noted that the errors of the military and 
general-aviation aircraft were generally larger than those of the civil 
transports . 
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trihnt^ r tb ® ta ke-off and landing-approach condition the error dis- 
trihutions are biased m a direction that would result from hysteresis 
aractenstics (aftereffect and recovery) and friction of the instrument 

t h n t P th S S v!?" 6 lag ° f the tubing-instrument system. Supporting evidence 
at the bias values of the two error distributions are actually repre- 

systems was derlved from the results 
of tests of 25 of the aircraft for which readings were obtained in both 

Se iSoi? ta f ■ ■?" c ° nditi ° ns - T»e average difference between 

the errors measured with these aircraft in the two conditions was 

fh/fV h . 1S i S ° f the Same 0rder as the sum of the bias values 
^5 feet) of the error distributions for the landing and take-off 
conditions. 


CONCLUDING REMARKS 


From the results of tests of 196 civil transport, military and 

heS of V 2S 1 fee^ rPl T e ? Sf 1 " 18 415 landing a PP roaches (at ^ average 
height of 280 feet) and of 70 airplanes in 152 take-off operations (at 

an average height of 440 feet), it has been determined that: 

1 . The overall error of the altimeter installations in the landing- 

coadltlon had a probable value (50 percent probability) of 

JL rl\ * “ST" 1 P robatle value ( 99.7 percent probability of 
±159 feet with a bias of +10 feet. 

n-r +kv'-p Th + ov ® ra11 error in the take-off condition had a probable value 
-55~flet eet ^ a maximUm P robable value of ±207 feet with a bias of 

3 - The overall errors of the military airplanes were cenerallv 
®J g ^. thaa thosa of the clvil transports in both the landing-approach 
error and°th COnditlons - In the landing-approach condition the probable 
and + 18 Q feet maximum Probable error of the military airplanes were ±45 
thf + ' ’ res P ectivel y> wlth a bias of +15 feet, whereas those for 

of +ffoot? ranSP ° rtS W6re ±22 and ±96 f8et " res P ec tively, with a bias 

, vH 4 ' The - bia ® values of the error distributions (+10 feet for the 

of the g h a ? d “ 3 ? fS ! t f ° r the take ; offs ) appear to represent a measure 
of the hysteresis characteristics (aftereffect and recovery) and friction 

of the instrument and the pressure lag of the tubing-instrJment system 


Langley Research Center, 

National Aeronautics and Space Administration 
Langley Field, Va., June 11, i960. 
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TABLE I.- RANGE OF OVERALL ALTIMETRY ERRORS 
OF THE VARIOUS TYPES OF AIRPLANES IN THE 
LANDING -APPROACH CONDITION 1 


(a) Civil transport airplanes 


Type of airplane 

Number of airplanes 

Range of errors, 
ft 

Airline A 

Tm -1 

8 

-46 to +11 

Tin - 2 

9 

-23 to +40 

Tm -3 

2 

-31 to +1 

Tm -4 

5 

-65 to +6 

Tm -5 

20 

-40 to +57 

Airline B 

Tm -3 

8 

-12 to +58 

Tm-6 

11 

-122 to +71 

Tm -7 

3 

-60 to +^3 

Airline C 

Tm -1 

1 

-34 

Tm-8 

7 

-99 to +68 


Pa 

POs 

POm 

TJ 

Tp 

Ts 

Tm 


Airplane-type designations: 
patrol, amphibious 

private-owner, small (2 to 3 passengers) 
private-owner, medium (4 to 5 passengers) 
trainer, jet powered 
trainer, propeller driven 
transport, small (6 to 11 passengers) 
transport, medium (21 to 52 passengers) 


Numerals following an aircraft designation represent different 
types of airplanes within the class designation; for example, Pa -1 
and Pa -2 are two different types of patrol, amphibious airplanes. 




TABLE I.- RANGE OF OVERALL ALTIMETRY ERRORS 
OF THE VARIOUS TYPES OF AIRPLANES IN THE 
LANDING-APPROACH CONDITION - Continued 1 


(b) Military airplanes 


Type of airplane 

Number of airplanes 

Range of errors, 
ft 

Service A 

Pa-l 

4 

-14 to +77 

Tm-3 

2 

+103 to +184 

Service B 

POm-3 

6 

-46 to +221 

Ts-3 

2 

-19 to +46 

Ts-4 

7 

L _ 

-222 to +69 

Service C 

P0m-7 

2 

-18 to +192 

Tj-1 

9 

+8 to +134 

Tm-1 

11 

-82 to +125 

Tm-3 

7 

-34 to +103 

Tm-6 

5 

-22 to +128 


Service D 


Pa-2 

4 

-62 to +116 

Tp-1 

2 

-39 to +98 

Ts-1 

14 

-269 to +156 

Ts-5 

6 

-60 to +46 

Tm-1 

1 

+21 

Tm-3 

3 

-48 to +76 

Tm-6 

6 

-66 to +163 


See footnote on first page of taole 


















TABLE I.- RANGE OF OVERALL ALTIMETRY ERRORS 
OF THE VARIOUS TYPES OF AIRPLANES IN THE 
LANDING-APPROACH CONDITION - Concluded 1 


(c) General aviation airplanes 


Type of airplane 

Number of airplanes 

Range of errors, 
ft 

Executive transports 

Ts-l 

i 

-56 

Ts-2 

2 

+52 to +111 

Tm-l 

1 

-63 

Tm-9 

2 

+1 to +30 

Private-owner 

P0s-1 

1 

-12 

P0s-2 

1 

+36 

P0s-3 

4 

-12 to +44 

P0s-4 

3 

+8 to +50 

P0s-5 

1 

+44 

P0s-6 

1 

+9 

P0s-7 

2 

-40 to +19 

P0s-8 

3 

-35 to +17 

P0m-1 

3 

-88 to +86 

P0m-2 

2 

+54 to +59 

P0m-3 

1 

-86 

P0m-4 

1 

-8 

P0m-5 

1 

+38 

P0m-6 

1 

+22 


1 See footnote on first page of table. 
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Figure 3.- Distribution of l 6 l altimetry errors of 74 civil transports 
as measured in 154 landing approaches. 
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Figure k.- Distribution of 2kQ altimetry errors of 91 military airplanes 
as measured in 226 landing approaches . 
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